Light-Field Imaging 101

A comprehensive introduction to unfocused light-field imaging (plenoptic 1.0)

Intended Audience

This guide aims to expand upon existing unfocused light-field (plenoptic 1.0) imaging learning material by providing detailed Python implementations of light-field image processing along with examples. The light-field image processing workflow closely follows that of the MATLAB-based Light-Field Imaging Toolkit [1], but leverages Python’s open-source ecosystem and Quarto’s enhanced capabilities for visualization and LaTeX formula integration.

By focusing specifically on the image processing aspects, this guide helps readers better understand the technical limitations and practical considerations of light-field imaging techniques. This guide assumes the reader is familiar with unfocused light-field imaging. Such an understanding can be achieved by browsing plenoptic.info or reading the unfocused light-field part of the Light-Field Camera Working Principles chapter (Pages 11-25) of the book Development and Application of Light-Field Cameras in Fluid Measurements [2].

Preprocessing

Raw images

To illustrate light-field image processing concepts, a synthetic unfocused light-field image was generated and is shown in Figure 1. The image describes the raw pixel intensities as recorded by an unfocused light-field camera. Every lenslet of the camera projects the incident radiance into a defocused intensity distribution pattern. This manifests as a spatially constrained blur where the light energy from each microlens is distributed across multiple sensor pixels rather than being concentrated at the expected conjugate position. The image was generated using a commercial ray tracing software (OpticStudio, ANSYS).

Figure 1: Synthetic unfocused light-field image. The hexagonal patterning in the image is characteristic of the projection process by the microlens array.

Calibration

The calibration process seeks to pinpoint the centroid of each lenslet’s corresponding sensor region. A calibration image is acquired with the main lens’s aperture reduced to a minimum. The resulting calibration image consist of an array of small bright spots as shown in Figure 2.

Figure 2: Synthetic light-field calibration image. Every bright spot is indicative of the corresponding lenslet centroid on the sensor.

The centroid identification process based on intensity peaks in an image is a standard image processing technique widely documented in the literature, not exclusive to light-field imaging. Since the synthetic calibration image contains no sensor noise, the calibration procedure has been intentionally simplified to emphasize conceptual clarity and streamline the explanation.

from skimage import morphology
from skimage.morphology import disk
from skimage.measure import label, regionprops

# Load calibration image
calibration_image = tifffile.imread('calibration.tif')

# Apply hard-coded threshold
binary_calibration = calibration_image > 10

# Perform a binary dilation to ensure coverage of the spots for the computation
# of the weighted centroid
opened_calibration = morphology.binary_dilation(binary_calibration, disk(5))

# Create labels 
labeled_calibration = label(opened_calibration)

# Apply regionprops
regions_calibration = regionprops(labeled_calibration, intensity_image=calibration_image)

# Compute weighted centroids location
centroids = [region.centroid_weighted for region in regions_calibration]

print(len(centroids))

2731

References

[1]

J. Bolan, E. Hall, C. Clifford, and B. Thurow, “Light-field imaging toolkit,” SoftwareX, vol. 5, pp. 101–106, 2016.

[2]

S. Shi and T. New, Development and application of light-field cameras in fluid measurements. Springer, 2023.